Display device and control method thereof

ABSTRACT

A display device and control method thereof, for controlling output of a sound signal corresponding to an image by controlling output of the image without an extra control operation by determining whether to suspend a sound signal based on deactivating an image corresponding to the sound signal.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. § 119to Korean Patent Application Nos. 10-2018-0124671 filed on Oct. 18,2018, and 10-2019-0083339 filed on Jul. 10, 2019, filed in the KoreanIntellectual Property Office, the disclosures of which are hereinincorporated by reference in their entireties.

BACKGROUND 1. Field

The disclosure relates to a display device and control method thereof,and more particularly, to a display device and control method thereoffor providing interactive board functions.

2. Discussion of Related Art

Display devices used in conference rooms may include a remote controlfunction.

Remote control refers to a function of software or an operating systemto execute a program of an external device as if the program is runningin a device that is remotely controlling the external device.

Using the remote control, the user may use a display device in aconference room to access an external device at a remote place to use aresource of the external device.

Once connected to the external device at the remote location using theremote control function of the display device, the user may reproducemultimedia content.

For example, to control sound being reproduced with the remote controlfunction, the user executes the remote control function to control anaudio player program of the external device or control an audio systemof the display device.

Such remote control may discontinue the ongoing conference in aconference-oriented environment, causing inconvenience to the user.Hence, there is a need for studies on devices for providing variousservices while maintaining continuity in conference.

SUMMARY

The disclosure provides a display device and control method thereof, forcontrolling output of a sound signal corresponding to an image bycontrolling output of the image without an extra control operation.

According to an embodiment, there is provided a display device includinga communication interface configured to receive, from an externaldevice, video data of video information displayed on the external deviceand audio data corresponding to the video data, a speaker, a display, aninput interface configured to receive user input of a user of thedisplay device, and a controller configured to control the display todisplay the video information, based on the video data, control thespeaker to output audio, based on the audio data, control the inputinterface to receive the user input to deactivate displaying of thevideo information, and control the speaker to suspend the audio, basedon the user input to deactivate the displaying of the video information.

According to an embodiment, there is provided a method of controlling adisplay device, the method including receiving, from an external device,video data of video information displayed on the external device andaudio data corresponding to the video data, controlling a display of thedisplay device to display the video information, based on the videodata, controlling a speaker of the display device to output audio, basedon the audio data, receiving user input to deactivate displaying of thevideo information, and controlling the speaker to suspend the audio,based on the user input to deactivate the displaying of the videoinformation.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the disclosurewill become more apparent to those of ordinary skill in the art bydescribing in detail exemplary embodiments thereof with reference to theaccompanying drawings, in which:

FIG. 1 is an exterior view of a display device, according to anembodiment of the disclosure;

FIG. 2 is a control block diagram of a display device, according to anembodiment of the disclosure;

FIG. 3 is a diagram illustrating a relationship between a display deviceand external devices, according to an embodiment of the disclosure;

FIG. 4 is a diagram illustrating the output of a sound signalcorresponding to an image of at least one area, according to anembodiment of the disclosure;

FIG. 5 is a diagram for explaining an operation to stop outputting soundbased on an image minimization command, according to an embodiment ofthe disclosure;

FIG. 6 is a diagram for explaining an operation to stop outputting soundby closing an image, according to an embodiment of the disclosure;

FIGS. 7 and 8 are diagrams for explaining an operation to set prioritiesamong applications, according to an embodiment of the disclosure;

FIGS. 9 and 10 are diagrams for explaining an operation performed when adisplay device is connected to a plurality of external devices,according to an embodiment of the disclosure; and

FIG. 11 is a diagram for explaining a display device performing anauthentication procedure and controlling an external device, accordingto an embodiment of the disclosure;

FIG. 12 is a diagram for explaining a display device performing anoperation based on a voice command;

FIG. 13 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure;

FIG. 14 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure;

FIG. 15 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure;

FIG. 16 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure; and

FIG. 17 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Like numerals refer to like elements throughout the specification. Notall elements of embodiments of the disclosure will be described, anddescription of what are commonly known in the art or what overlap eachother in the embodiments will be omitted. The terms as used throughoutthe specification, such as “˜ part”, “˜ module”, “˜ member”, “˜ block”,etc., may be implemented in software and/or hardware, and a plurality of“˜ parts”, “˜ modules”, “˜ members”, or “˜ blocks” may be implemented ina single element, or a single “˜ part”, “˜ module”, “˜ member”, or “˜block” may include a plurality of elements.

It will be further understood that the term “connect” or its derivativesrefer both to direct and indirect connection, and the indirectconnection includes a connection over a wireless communication network.

The term “include (or including)” or “comprise (or comprising)” isinclusive or open-ended and does not exclude additional, unrecitedelements or method steps, unless otherwise mentioned.

Throughout the specification, when it is said that a member is located“on” another member, it implies not only that the member is locatedadjacent to the other member but also that a third member exists betweenthe two members.

It will be understood that, although the terms first, second, third,etc., may be used herein to describe various elements, components,regions, layers and/or sections, these elements, components, regions,layers and/or sections should not be limited by these terms. These termsare only used to distinguish one element, component, region, layer orsection from another region, layer or section.

It is to be understood that the singular forms “a,” “an,” and “the”include plural references unless the context clearly dictates otherwise.

Reference numerals used for method steps are just used for convenienceof explanation, but not to limit an order of the steps. Thus, unless thecontext clearly dictates otherwise, the written order may be practicedotherwise.

The principle and embodiments of the disclosure will now be describedwith reference to accompanying drawings.

FIG. 1 illustrates a display device 100 and an external device 200,according to an embodiment of the disclosure.

Referring to FIG. 1, the display device 100 may communicate with atleast one external device 200. The external device 200 may beimplemented by various types of electronic devices such as a desktoppersonal computer (PC), a set top box, a smart phone, a tablet, etc. Thedisplay device 100 may be controlled to display a screen 200-1corresponding to the external device 200 (see FIG. 4). For example,while in communication with a PC, the display device 100 may display ascreen received from the PC.

The display device 100 may also display a screen 100-1 that is generatedby the display device 100 itself. The screen 100-1 corresponding to thedisplay device 100 may be a screen for executing the display device's100 own functions and/or applications.

For example, when the display device 100 is implemented as aninteractive board, the display device 100 may display a screen thatprovides an interactive board function. The interactive board functionmay provide a black or white screen function mimicking the functionalityof a conventional blackboard and a writing function. The screen is not,however, limited thereto, and a screen for executing any function thatis performed in an ordinary interactive board may be displayed.

Furthermore, as shown in FIG. 4, the display device 100 may display boththe screen corresponding to the display device 100 and the screencorresponding to the external device 200 at the same time. Specifically,the display device 100 may display the screen corresponding to thedisplay device 100 in some display area and the screen corresponding tothe external device 200 in the other display area. The display device100 is not, however, limited thereto, and the display device 100 maydisplay just one screen corresponding to the external device 200 or thedisplay device 100.

Furthermore, the display device 100 may include a sensor for detectingtouches, for example according to a touch received on a touch screendisplay. When detecting a touch, the sensor may send informationcorresponding to the touch to an external device or the display devicebased on the touch type. The touch type will be described later indetail.

FIG. 2 is a block diagram of the display device 100, according to anembodiment of the disclosure.

Referring to FIG. 2, the display device 100 may include a display 110, aspeaker 120, an input 130, a communicator 150, a storage 160, and acontroller 140.

The display device 100 may be implemented with various types ofelectronic devices such as a television, an interactive board, anelectronic table, a large format display (LFD), etc. The interactiveboard may refer to an electronic device equipped with a touch screenpanel implemented in e.g., an infrared touch screen type using aninfrared (IR) sensor, a surface acoustic wave touch screen type, aresistive touch screen type, a capacitive touch screen type, etc., toallow writing thereon with an electronic pen or the finger of the user.The display device 100 is not, however, limited thereto, and theinteractive board may also be implemented in a way of identifying atouch area using e.g., a camera.

Furthermore, in an embodiment, the display device 100 may be implementedas the interactive board and also as a television, electronic table, orLFD capable of performing the interactive board function.

The display 110 may display at least one of the screen of the displaydevice 100 itself and a screen received from an external device.

Furthermore, the display 110 may display various contents, such as allkinds of moving images created as a plurality of images, such as a film,a soap opera, a recorded image, a slow video, etc., and displayed withcertain frame ratios, still images, photos, documents, etc.

The display 110 may also display a user interface screen including manydifferent menus provided by applications or an operating system of thedisplay device 100. The user interface screen may include various menusto control operations of the display device 100, e.g., volume up/downmenus, channel up/down menus, setting menus, etc.

Especially, the display 110 may display a screen that provides theinteractive board function. The screen that provides the interactiveboard function may be displayed with a black or white background image.The background image may be provided as a transparent screen withadjusted transparency. The interactive board function refers to allfunctions that may be performed in an interactive board device.

The display 110 may be implemented with any display panel such as aliquid crystal display (LCD), organic light emitting diodes (OLEDs), aplasma display panel (PDP), or the like.

The speaker 120 may be implemented for outputting sound. The speaker 120may output a sound signal corresponding to an image, as will bedescribed later. The speaker 120 may output a sound signal of thedisplay device 100 or of an external device remotely coupled to thedisplay device 100.

The input 130 may be an interface implemented to receive a command ofthe user. Specifically, the input 130 may include an interface forreceiving signals from a remote controller, an interface for receivinginput via a touch panel, a mechanical button mounted on the displaydevice 100, a keyboard, a mouse, etc.

When the display 110 is touch-sensitive, the input 130 may detect atouch against the display 110. Specifically, the input 130 may includevarious types of sensors. For example, the input 130 may be implementedas a touch sensor to recognize touches, a motion sensor to detect motionof the user, or a sensor to detect movements and trajectories of anelectronic pen.

The user may activate or deactivate a portion of an image reproduced onthe display 110 through the input 130. The user may change priority ofan image of at least one area output on the display 110 through theinput 130. The user may deactivate an image by entering a command tominimize an image of at least one area output on the display 110 throughthe input 130.

The communicator 150 may be a communication interface configured toperform communication with at least one external device. Specifically,the display device 100 may be wired or wirelessly connected to anexternal device through the communicator 150. The communicator 150 maythen transmit to or receive data from the external device via auniversal serial bus (USB), a serial port, Wireless Fidelity (Wi-Fi),Bluetooth, etc. The supported standard is not limited thereto, and otherstandards may also be supported. Accordingly, the display device 100 mayuse the communicator 150 to receive data and display a screencorresponding to the connected external device.

The communicator 150 may include one or more components that enablecommunication with an external device, for example, at least one of ashort-range communication module, a wired communication module, and awireless communication module.

The short-range communication module may include various short rangecommunication modules for transmitting and receiving signals within ashort range over a wireless communication network, such as Bluetoothmodule, an infrared communication module, a radio frequencyidentification (RFID) communication module, a wireless local accessnetwork (WLAN) communication module, a near field communication (NFC)module, a Zigbee communication module, etc.

The wired communication module may include not only one of various wiredcommunication modules, such as a network interface controller (NIC) forcommunicating over a network such as the Internet, a controller areanetwork (CAN) communication module, a local area network (LAN) module, awide area network (WAN) module, or a value added network (VAN) module,but also one of various cable communication modules, such as a universalserial bus (USB), a high definition multimedia interface (HDMI), adigital visual interface (DVI), recommended standard (RS) 232, a powercable, or a plain old telephone service (POTS).

The wireless communication module may include a wireless fidelity (WiFi)module, a wireless broadband (Wibro) module, and/or any wirelesscommunication device for supporting various wireless communicationschemes, such as a global system for mobile communication (GSM) module,a code division multiple access (CDMA) module, a wideband code divisionmultiple access (WCDMA) module, a universal mobile telecommunicationssystem (UMTS), a time division multiple access (TDMA) module, a longterm evolution (LTE) module, etc.

The storage 160 may be memory that stores various data, programs, orapplications for driving and controlling the display device 100.

The storage 160 may store a control program to control the displaydevice 100 when the control program is executed by a processor such asthe controller 140, an application provided by a manufacturer by defaultor downloaded from an external source, a graphical user interface (GUI)related with the application, objects to provide the GUI, e.g., images,text, icons, buttons, etc., user information, documents, databases, orassociated data.

The storage 160 may be implemented as a memory card such as micro securedigital (SD) card, a universal serial bus (USB) memory, or the like, anon-volatile memory, a volatile memory, a hard disc drive (HDD), or asolid state drive (SSD).

When the display device 100 is implemented as an interactive board, thestorage 160 may store a computer program to perform the interactiveboard function when executed under control of the controller 140. Toprovide a screen to perform the interactive board function, the programstored in the storage 160 may be executed by a processor such as thecontroller 140. The program to perform the interactive board may bestored at a design stage of the display device 100, and updated in theform of firmware. Such updating in the form of firmware may be performedby wired or wireless communication through the communicator 150.

The controller 140 may control general operations of the display device100. The controller 140 may be implemented as a microprocessor with amemory for storing an algorithm to control operation of the componentsof the display device 100 or data about a program that embodies thealgorithm, and a processor for carrying out the aforementioned operationusing the data stored in the memory. The memory and the processor may beimplemented in separate chips. Alternatively, the memory and theprocessor may be implemented in a single chip.

The controller 140 may access the storage 160 to execute an operatingsystem (OS) stored in the storage 160 for providing general operationsof the display device 100. The controller 140 accesses various programs,contents, data, etc., stored in the storage 160 to perform variousoperations of the display device 100.

The controller 140 may determine whether to output a sound signalcorresponding to an image of at least one area based on a state of theimage, based on a command from the user.

The image of the at least one area may refer to images output in windowson the display 110. Furthermore, the image of the at least one area maybe an image output from each application, or an image output from eachexternal device, which will be described later. The at least one areamay be displayed as including a visibly displayed boundary.

The state of an image may include ‘activated’ and ‘deactivated’. Anactivated image may include that the user determines to watch the image.For example, when an image of at least one area is output with higherpriority than others, or when the image is output in full screen, theuser controls the image or an application including the image.

A deactivated image may include that the user does not watch the image.However, deactivating an image does not mean turning off the imagecompletely, but maintaining the image ready to be activated at any timeaccording to a command from the user.

For example, an image may be deactivated e.g., when the image isminimized, when the user selects an area other than the area of theimage to run an application other than the application corresponding tothe image, when the user eliminates the image but generates the image inthe background, when the user gives second priority to the image in manydifferent ways, etc.

The controller 140 may control output of a sound signal corresponding toan image of at least one area to be stopped by deactivating the image ofthe at least one area output on the display 110, based on a command fromthe user.

Deactivating an image of at least one area may be based on an operationof the user changing priority, a command from the user to minimize theimage, and a command from the user to select an area on the displayother than the at least one area.

An image of at least one area output on the display 110 may be imagesresulting from a plurality of applications output on the display device100. In this case, the controller 140 may control output of a soundsignal from an application to be stopped by deactivating the imageoutput from the application.

An image of at least one area reproduced on the display 110 may be animage of a screen provided by an external device connected to thedisplay device 100. In this case, when the image of the at least onearea is deactivated, the controller 140 may control the display 110 notto output the sound signal received from the external device.

Furthermore, the controller 140 may control creation or output of asound signal in the external device to be stopped by sending a controlsignal to stop the sound signal to the external device. In this case,the control signal may be different depending on the external device.Specifically, the controller 140 may create a control signal based on akind of the OS executed by the external device. This will be describedlater in detail.

The controller 140 may activate output of a sound signal of an externaldevice, which corresponds to an image selected from among images of theplurality of external devices, and stop outputting sound signals of theother external devices corresponding to non-selected images. The displaydevice 100 may be connected to the plurality of external devices and maysimultaneously output the plurality of images output from the externaldevices. The user may select one of the plurality of images through theinput 130 to output a sound signal corresponding to the selected image.

At this time, the controller 140 may stop outputting sound signalscorresponding to other images or may reduce volume of such sound signalsto be inaudible. Therefore, audio interference between competing soundsignals may be avoided.

The controller 140 may stop outputting the sound signal by omitting theimage from the display 110. Stopping output of a sound signal byomitting an image may be similar to an operation of stopping the soundsignal by minimizing the image. Alternatively, sound may be stopped whena corresponding image is closed or a connection to the external deviceproviding the image is suspended or terminated.

Furthermore, the controller 140 may release suspension of the soundsignal by activating the deactivated image of at least one area.Specifically, the controller 140 may quickly activate or deactivate animage based on a command from the user, and accordingly activate ordeactivate a sound signal corresponding to the image.

For example, an occasion is assumed when the display device 100 isimplemented as an interactive board and the display 110 displays ascreen that provides interactive board functions. In this case, thecontroller 140 may execute an interactive board function displayed atthe coordinates of a detected touch point. The interactive boardfunction may include a writing (input) function, an editing function, avideo play function, a function for storing a screen on which a writingis input, etc.

The writing function may be performed based on an input of the user'stouch or an electronic pen, and the controller 140 may execute afunction or application corresponding to the input.

The editing function may include e.g., deleting, correcting or copyinginput text, or changing font size, font type, color, writing style,etc., and may include other commonly-used document editing functions.

The video play function refers to a function to reproduce moving images,still images, etc., on the interactive board.

The interactive board functions may not, however, be limited thereto,and may include all the other functions usable by the interactive board.

At least one component may be added or omitted to correspond to theperformance of the components of the display device 100 shown in FIG.100. Furthermore, as understood to those of ordinary skill in the art,the components of the display device 100 may be changed to correspond tothe system performance or structure.

The functions of the components of the display device 100 shown in FIG.2 may be implemented in software, or hardware such as Field ProgrammableGate Arrays (FPGAs) and Application Specific Integrated Circuits(ASICs).

FIG. 3 shows a relationship between a display device and externaldevices 200 and 210, according to an embodiment of the disclosure.

Referring to FIG. 3, the display device 100 may be connected to at leastone external device to communicate with the external device 200, 210.

The display device 100 may communicate with the at least one externaldevice 200 and 210. The display device 100 may communicate with theexternal device directly or over one or more networks.

In an embodiment of the disclosure, the network may include a local areanetwork (LAN). The display device 100 may receive a signal from theexternal device 200 or 210 and display an image output from the externaldevice. Receiving a signal from the external device 200 or 210 at thedisplay device 100 may be performed by the user through remote control.In an embodiment of the disclosure, the user of the display device 100may remotely control an external device to reproduce multimedia contentof the external device.

In an embodiment of the disclosure, the display device 100 may determinea state of remote control operation and accordingly transmit aparticular control signal to the external device 200 or 210.

Specifically, the control signal transmitted by the display device 100to the external device 200 or 210 may include at least one key code.

The key code transmitted by the display device 100 may be created orchanged based on the operating system (OS) executed by each of thedisplay device 100 and the external devices 100 and 210.

For example, the windows OS may use a virtual key code (VK) scheme, theLinux OS an x key code (XK) scheme or xfree86 key code (XF86) scheme,and the Mac OS an Apple VK scheme.

When the display device 100 transmits a control signal to the externaldevice 200 or 210, the display device 10 may modify and transmit thecontrol signal usable by the external device 200 or 210. The displaydevice 100 may change the key code included in the control signal basedon the OS executed by the external device. In an embodiment of thedisclosure, the key codes VK, XK, XF86, and Apple VK may beinterchangeably used. In an embodiment of the disclosure, when thedisplay device 100 may operate in the Linux OS, the controller 140 ofthe display device 100 may create the control signal with the XK codeand change the control signal to one including VK, XF86, or Apple VKcode based on the OS of the external device.

The display device 100 may create a control signal to control theexternal device 200 or 210 and transmit the control signal to theexternal device 200 or 210.

In an embodiment of the disclosure, the display device 100 may transmita control signal to the external device 200 or 210, which includes a keycode to instruct the external device 200, 210 to stop outputting soundof the external device 200 or 210. Specifically, the external device 200or 210 may include a sound output controller for controlling soundoutput, which may stop outputting and/or transmission of a sound signalof the external device upon reception of the control signal from thedisplay device 100.

The configuration of FIG. 3 is merely an example, and there are nolimitations on how the display device 100 may control suspension of asound signal of the external device.

FIG. 4 shows outputting a sound signal corresponding to an image of atleast one area, according to an embodiment of the disclosure, and FIG. 5is a diagram for explaining an operation to stop outputting sound basedon an image minimization command, according to an embodiment of thedisclosure.

Referring to FIG. 4, as described above, the display device 100 maycommunicate with at least one external device 200. The display device100 may be connected to the external device 200 directly or over anetwork. The display device 100 may remotely control the external device200.

In an embodiment of the disclosure, the user may remotely control theexternal device 200 through the display device 100 to have an image200-1 or sound output by the external device 200 to be reproducedthrough a screen 100-1 of the display device 100 or to stop outputtingthe image 200-1 or the sound. In other words, the user may enter asignal to reproduce an image output by the external device 200, on thedisplay device 100, or to stop reproducing the image on the displaydevice 100.

Furthermore, the display device 100 may output or stop outputting asound signal provided from the external device 200, at the displaydevice 100. In an embodiment of the disclosure, the user may enter acommand to silence, stop, suspend, or otherwise make inaudible the soundof the external device 200 through the display device 100.

FIG. 5 is a diagram for explaining an operation to stop outputting soundbased on an image minimization command, according to an embodiment ofthe disclosure.

Referring to FIG. 5, an image of the external device 200 that has beendisplayed in the display device 100 is minimized 200-2. Minimizing 200-2an image of the external device 200 may be performed through the input130, or through at least one button equipped in the display device 100.

The operation of minimizing 200-2 an image of the external device 200 atthe display device 100 may include activating and deactivating a remotecontrol function.

In this case, deactivation refers to minimization that maintains thefunction ready to be activated at any time based on another command fromthe user. When the image provided by the external device 200 isminimized 200-2, the reproduction of the image provided by the displaydevice 100 is suspended or otherwise obscured from a user of the displaydevice 100.

When the image of the external device 200 is minimized 200-2 at thedisplay device 100, the controller 140 may transmit a signal to controla sound signal of the external device 200 based on the state of theimage of the external device 200. Specifically, when the image of theexternal device 200 is minimized, the controller 140 may transmit acontrol signal including a key code to silence the sound signal of theexternal device 200 to the external device 200. Based on the key codetransmitted from the display device 100, the external device 200 maystop outputting the sound signal and/or transmitting the sound signal.

FIG. 6 is a diagram for explaining an operation to stop outputting soundby removing an image, according to an embodiment of the disclosure.

Referring to FIG. 6, an image of the external device 200 that has beendisplayed at the display device 100 is removed 200-1. Removing 200-1 animage of the external device 200 may be performed through the input 130,or through at least one button equipped in the display device 100. Theremoving 200-1 of an image from the display device 100 may includestopping execution of an application corresponding to the image, butmaintaining general execution of the application to be activated at anytime based on a command from the user.

The removal 200-1 of an image from the display device 100 may also bebased on activating and deactivating the remote control function.

In this case, deactivation refers to minimization that maintains thefunction ready to be activated at any time based on another command fromthe user.

When the image of the external device 200 is removed from display of thedisplay device 100, the controller 140 may transmit a signal to controla sound signal of the external device 200 based on the state of theimage of the external device 200. Specifically, when the image of theexternal device 200 is removed from the display device 100, thecontroller 140 may transmit a control signal including a key code tosilence the sound signal of the external device 200 to the externaldevice 200. Based on the key code transmitted from the display device100, the external device 200 may stop outputting the sound signal.

FIGS. 5 and 6 are examples of deactivating an image output by thedisplay device, and there are no limitations on how to deactivate theimage.

FIGS. 7 and 8 are diagrams for explaining an operation to set prioritiesamong applications, according to an embodiment of the disclosure.

Referring to FIG. 7, images 100-2 and 200-1 correspond to an applicationexecuted by the display device 100 itself and an application executed byan external device 200, respectively, are shown on the screen 100-1 ofthe display device 100. The user may execute the application at thedisplay device 100 itself, and simultaneously control the image 200-1 ofthe external device to be provided by remote control. The user maycontrol output of a sound signal corresponding to an image driven at thedisplay device 100 or output of a sound signal corresponding to theimage 200-1 of the external device, based on a selection of the useraccording to user input through the input 130.

In this case, the user may set a priority for each image. In anembodiment of the disclosure, the user may assign top priority to animage the user wants to watch by placing the image 100-2 on the toplayer of the screen 100-1. The display device 100 may output a soundsignal corresponding to the image given top priority by the user. InFIG. 7, the image 100-2 originated from the display device 100 has thetop priority and the image 200-1 originated from the external device 200has a lower priority. In this case, the display device 100 may output asound signal corresponding to the image 100-2 originated from thedisplay device 100, while not providing the sound of the external device200 based on the aforementioned operations.

On the other hand, the user may assign the top priority to the imageoriginated from the external device 200, in which case the displaydevice 100 may not provide the sound signal of the application runningat the display device 100, and instead provide sound of the applicationexecuted by the external device 200.

Referring to FIG. 8, an operation of setting mutual priority betweenapplications running at the display device 100 will be described. Thecontroller 114 of the display device 100 may execute a plurality ofapplications 100-2 and 100-3, at the display device 100. When at leastone of the plurality of applications outputs sound via the displaydevice 100, the user may set a top priority to one of the plurality ofapplications, e.g., 100-2, to control the sound of the application giventhe top priority to be output through the display device 100. The toppriority may be given for an application by placing the application onthe top layer on the display device 100 as shown in FIG. 7, or by aselection that reflects intention of the user as shown in FIG. 8. Forexample, in the display device 100, a first application and a secondapplication may run at the same time.

When the user selects the first application, e.g., 100-2, with the toppriority by selection through the input 130, the controller 114 maycontrol the sound of the first application to be output through thedisplay device 100 while controlling the sound of the secondapplication, e.g., 100-3, to not be provided by the display device 100.Selecting the application by the user may be included in the operationof activating the application.

Furthermore, the user may silence the sound of the image 100-2, 100-3 byselecting an area 100-1 other than the plurality of application areas100-2 and 100-3. Selecting the area 100-1 other than the area of animage may indicate that the user has little intention to watch theeither image, so the sound corresponding to both images 100-2, 100-3 maynot be provided.

Although an application is selected to be activated in FIGS. 7 and 8,activation of applications is not limited thereto.

FIGS. 9 and 10 are diagrams for explaining an operation performed when adisplay device is connected to a plurality of external devices,according to an embodiment of the disclosure.

Referring to FIGS. 9 and 10, the display device 100 communicates withthe plurality of external devices 200 and 210. The plurality of externaldevices 200 and 210 may include the first external device 200 and thesecond external device 210. Although there are two external devicesshown in FIGS. 9 and 10, the number of external devices is merely anexample and there are no limitations on the number of external devicesthat may be connected to the display device 100. Furthermore, the terms‘first’ and ‘second’ used for the external devices are to distinguishone external device from another without meaning to assign priority.

Referring to FIG. 9, the display device 100 may output images 200-1 and200-2 originated from both the first and second external devices 200 and210.

The user may activate sound output from the external device 200corresponding to the image 200-1 by selecting the image 200-1 of theexternal device 200. At the same time, the controller 114 of the displaydevice 100 may stop the external device 210 from outputting soundsignals corresponding to the image 210-1 not selected.

How the user selects an image originated from an external device may beembodied as shown in FIG. 9. Specifically, when the user selects theimage 200-1 originated from the first external device 200 at the displaydevice 100, the display device 100 may transmit a control signal tocontrol sound output to the first external device 200. The displaydevice 100 may transmit another control signal to stop sound output tothe unselected second external device 210.

The display device 100 may output the sound corresponding to the firstexternal device 200 and withhold outputting of the sound correspondingto the second external device 210.

While the display device 100 outputs the sound signal of the firstexternal device 200 that corresponds to an image that the user wants toactivate and suspends outputting the sound signal of the second externaldevice 200 not selected as shown in FIG. 9, the sound output may bestopped by selection, as will be described later.

Referring to FIG. 10, the display device 100 may be connected to thefirst external device 200 and the second external device 210 as in FIG.9. The user may minimize an image 200-1, 210-1 corresponding to thefirst external device 200 or second external device 210. In FIG. 10, theuser minimizes an image 210-2 corresponding to the second externaldevice 210. The display device 100 may stop outputting a sound signalfrom the second external device 210 corresponding to the minimized image210-2. Specifically, the display device 100 may transmit a controlsignal to stop outputting the sound from the second external device 210.As described above, the control signal transmitted by the display device100 may include a corresponding key code. Furthermore, when the displaydevice 100 is connected to a plurality of external devices, the controlsignal for transmission may be changed based on the OS used by theconnected external device. Transmission of a control signal based on theOS was described above, so the description thereof will not be repeated.

FIG. 11 is a diagram for explaining a display device performing anauthentication procedure and controlling external devices 220 and 230,according to an embodiment of the disclosure.

The controller 114 may communicate with at least one external device 220and 230 after performing a predefined authentication and/or pairingprocedure with the external device based on a command from the user.

In general, the authentication procedure may refer to a process for adevice to identify unique information of another device before thedevice is connected to the other device to control the other device.

In general, during an authentication procedure, the external device 220or 230 may request a name, identity (ID), and password of the user orthe display device 100.

In the embodiments of the disclosure, the display device 100 may have anapplication related to the authentication procedure installed thereon inadvance.

The external device 220 or 230 may also have an application installed tocorrespond to the application installed in the display device 100. Theapplications installed in the display device 100 and the external device220 or 230 may automatically perform a predefined authenticationprocedure. This may allow the user to perform the authenticationprocedure without entering an extra password when the user is to controlthe external device 220 or 230 through the display device 100.

The aforementioned authentication procedure is merely an example of apredefined authentication procedure, without being limited thereto.

The controller 114 may create a signal to control the at least oneexternal device through the display device 100 based on a command fromthe user.

Specifically, the controller 114 may convert an input from the user to acontrol signal that an OS protocol supported by each of the at least oneexternal device 220 or 230.

The controller 114 may send the control signal to the external deviceover a network connected to the display device 100 and the externaldevice 220 or 230.

Accordingly, the controller 114 of the display device 100 may control aremote device based on each control signal.

For example, when the user enters a command to deactivate an image of atleast one area, the controller 114 may create a control signalcorresponding to the OS (e.g., Windows®, Linux®, IOs®, Android®, etc.)of each external device 220 or 230 based on the command.

Specifically, the controller 114 may convert the command into a key coderelated to audio silence among virtual keyboard key codes used in theprotocol of each external device, e.g., rdp, vnc, etc. The controller114 may send the key code to the external device over the network. Afterthe external device receives the key code, sound signals of the externaldevice may be remotely controlled.

The operation as described above in connection with FIG. 11 is merely anexample, and there are no limitations on how the user controls theexternal device through the display device 100.

FIG. 12 is a diagram for explaining a display device performing anoperation based on a voice command.

Referring to FIG. 12, the user's command may include a voice command Vas user input.

The controller 114 may create a control signal to control at least oneof the display device 100 and the at least one external device based onthe voice command V.

Specifically, when the user enters the voice command V, the controller114 may convert the voice command to a command used in the protocol ofan OS of each external device.

In this case, the input 130 may include a microphone.

For example, the user may enter a command by saying “turn down thesound” through the microphone.

The controller 114 may include a voice recognition application.

The controller 114 may convert the user's voice command to a key coderelated to an audio process among the virtual keyboard key codes used inthe protocol of each external device.

The controller 114 may send the key code to the external device over thenetwork, and after the external device receives the key code, an audiofunction of the external device may be controlled. In this embodiment,an external device is controlled based on the voice command, but in someother embodiments, the display device 100 may also be controlled basedon the voice command.

The operation as described in connection with FIG. 12 is merely anexample, and there are no limitations on how the user controls thedisplay device or the external device based on the voice command.

FIG. 13 is a flowchart of a method of controlling a display apparatus,according to an embodiment of the disclosure, FIG. 14 is a flowchart ofa method of controlling a display apparatus, according to an embodimentof the disclosure, FIG. 15 is a flowchart of a method of controlling adisplay apparatus, according to an embodiment of the disclosure, FIG. 16is a flowchart of a method of controlling a display apparatus, accordingto an embodiment of the disclosure, and FIG. 17 is a flowchart of amethod of controlling a display apparatus, according to an embodiment ofthe disclosure.

Referring to FIG. 13, the user enters a command to deactivate an imagethrough the input 130 of the display device 100, in 1001. The controller114 stops outputting a sound signal corresponding to the image, in 1002,in response to the command.

Referring to FIG. 14, an occasion of remote control through the displaydevice 100 is described. The display device 100 communicates with atleast one external device, in 1011. The display device 100 deactivatesan image corresponding to the external device based on a command fromthe user, in 1012. The deactivation of the image may include minimizingthe image. The display device 100 creates a control signal correspondingto the deactivation of the image, in 1013. The control signal mayinclude a key code. The display device 100 modifies the control signalbased on an OS used by the external device, in 1014. The modifiedcontrol signal is transmitted to the external device, in 1015. Whenreceiving the control signal, the external device stops outputting asound signal based on the control signal, in 1016.

FIG. 15 is a flowchart of a method when the display device 100communicates with a plurality of external devices.

The display device 100 communicates with a plurality of externaldevices, in 1021. The user of the display device 100 selects an imagecorresponding to an external device, in 1022. The display device 100 mayactivate and output a sound signal corresponding to the selected imageand deactivate and stop outputting a sound signal corresponding to anunselected image. To execute this control, the display device 100creates corresponding control signals, in 1023, and transmits thecontrol signals to the devices selected and not selected based on theselected image, in 1024. An external device corresponding to theunselected image receives the control signal to stop outputting thesound, in 1025, and ceases to output the sound.

Referring to FIG. 16, the user enters a command, in 1031. Based on thecommand from the user, the display device 100 performs an authenticationprocedure with an external device, 1032. After completion of theauthentication procedure between the display device 100 and the externaldevice, the user remotely controls the external device through thedisplay device 100, in 1033.

Referring to FIG. 17, the user enters a voice command, in 1041. Thedisplay device 100 converts the voice command into a control signal, in1042. The display device 100 controls the display device 100 itself orthe external device based on the control command, in 1043.

The embodiments of the disclosure may be implemented in the form ofrecording media for storing instructions of computer programs to beexecuted by a computer. The computer-readable instructions may be storedin the form of program codes, and when executed by a processor, mayconstitute program modules to perform operations in the embodiments ofthe disclosure. The recording media may correspond to computer-readablerecording media.

The computer-readable recording medium includes any type of recordingmedium having data stored thereon that may be thereafter read by acomputer. For example, the computer-readable recording medium may be aROM, a RAM, a magnetic tape, a magnetic disk, a flash memory, an opticaldata storage device, etc.

According to an embodiment of the disclosure, a display device andcontrol method thereof may control output of a sound signalcorresponding to an image by controlling image output without an extracontrol operation.

Several embodiments have been described above, but a person of ordinaryskill in the art will understand and appreciate that variousmodifications can be made without departing the scope of the disclosure.Thus, it will be apparent to those ordinary skilled in the art that thetrue scope of technical protection is only defined by the followingclaims.

What is claimed is:
 1. A display device comprising: a communicationinterface configured to receive, from an external device, video data ofvideo information displayed on the external device and audio datacorresponding to the video data; a speaker; a display; an inputinterface configured to receive user input of a user of the displaydevice; and a controller configured to: control the display to displaythe video information, based on the video data, control the speaker tooutput audio, based on the audio data, control the input interface toreceive the user input to deactivate displaying of the videoinformation, and control the speaker to suspend the audio, based on theuser input to deactivate the displaying of the video information.
 2. Thedisplay device of claim 1, wherein the controller is further configuredto, when the input interface receives the user input to deactivate thedisplaying of the video information, control the speaker to stop theaudio.
 3. The display device of claim 1, wherein the controller isfurther configured to determine whether to output the audiocorresponding to the video data based on a state of the videoinformation based on the user input of the user.
 4. The display deviceof claim 1, wherein the controller is further configured to determinewhether to output the audio by transmitting at least one control signalbased on the user input to deactivate the displaying of the videoinformation to the external device.
 5. The display device of claim 1,wherein the controller is further configured to generate at least onecontrol signal based on an operating system used by the external device.6. The display device of claim 1, wherein the user input to deactivatethe displaying of the video information comprises a user input todisplay second video information of second video data received from asecond external device, and wherein the controller is further configuredto control the speaker to output second audio corresponding to thesecond video data, based on the user input to display the second videoinformation and stop outputting of the audio based on the audio datareceived from the external device.
 7. The display device of claim 1,wherein the user input to deactivate the displaying of the videoinformation comprises a command to change display priority of the videoinformation received from the external device, and wherein thecontroller is further configured to control the speaker to suspend theaudio, based on the priority of the video information.
 8. The displaydevice of claim 1, wherein the user input to deactivate the displayingof the video information comprises a command to select a second area onthe display other than a first area on which the video information isdisplayed, and wherein the controller is further configured to controlthe speaker to suspend the audio, based on the command to select thesecond area on the display.
 9. The display device of claim 1, whereinthe user input to deactivate the displaying of the video informationcomprises a command to minimize an area on which the video informationis displayed, and wherein the controller is further configured tocontrol the speaker to suspend the audio, based on the command tominimize the area.
 10. The display device of claim 1, wherein the userinput to deactivate the displaying of the video information comprises acommand to remove the video information from the display, and whereinthe controller is further configured to control the speaker to suspendthe audio based on the command to remove the video information from thedisplay.
 11. The display device of claim 1, wherein the controller isfurther configured to control the input interface to receive user inputto activate displaying of the video information and control the speakerto release suspension of the audio based on the user input to activatethe displaying of the video information.
 12. The display device of claim1, wherein the controller is further configured to communicate with theexternal device after performing a predefined authentication procedurewith the external device.
 13. The display device of claim 12, whereinthe controller is configured to create a control signal to control theexternal device through the display device.
 14. The display device ofclaim 1, wherein the user input comprises a voice command.
 15. A methodof controlling a display device, the method comprising: receiving, froman external device, video data of video information displayed on theexternal device and audio data corresponding to the video data;controlling a display of the display device to display the videoinformation, based on the video data; controlling a speaker of thedisplay device to output audio, based on the audio data; receiving userinput to deactivate displaying of the video information; and controllingthe speaker to suspend the audio, based on the user input to deactivatethe displaying of the video information.
 16. The method of claim 15,wherein the controlling the speaker to suspend the audio comprisescontrolling the speaker to suspend the audio when the user input todeactivate the displaying of the video information is received.
 17. Themethod of claim 15, wherein the controlling the speaker to suspend theaudio comprises: determining whether to output the audio correspondingto the video data based on a state of the video information based on theuser input.
 18. The method of claim 15, wherein the controlling thespeaker to suspend the audio comprises: determining whether to outputthe audio by transmitting at least one control signal based on the userinput to deactivate the displaying of the video information to theexternal device.
 19. The method of claim 15, wherein the controlling thespeaker to suspend the audio comprises: generating at least one controlsignal based on an operating system used by the external device.
 20. Themethod of claim 15, wherein the user input to deactivate the displayingof the video information comprises a user input to display second videoinformation of second video data received from a second external device,and wherein the method further comprises: controlling the speaker tooutput second audio corresponding to the second video data, based on theuser input to display the second video information; and stoppingoutputting of the audio based on the audio data received from theexternal device.
 21. The method of claim 15, wherein the user input todeactivate the displaying of the video information comprises a commandto change display priority of the video information received from theexternal device, and wherein the controlling the speaker to suspend theaudio comprises controlling the speaker to suspend the audio, based onthe priority of the video information.
 22. The method of claim 15,wherein the user input to deactivate the displaying of the videoinformation comprises a command to select a second area on the displayother than a first area on which the video information is displayed, andwherein the controlling the speaker to suspend the audio comprisescontrolling the speaker to suspend the audio, based on the command toselect the second area on the display.
 23. The method of claim 15,wherein the user input to deactivate the displaying of the videoinformation comprises a command to minimize an area on which the videoinformation is displayed, and wherein the controlling the speaker tosuspend the audio comprises controlling the speaker to suspend theaudio, based on the command to minimize the area.
 24. The method ofclaim 15, wherein the user input to deactivate the displaying of thevideo information comprises a command to remove the video informationfrom the display, and wherein the controlling the speaker to suspend theaudio comprises controlling the speaker to suspend the audio based onthe command to remove the video information from the display.
 25. Themethod of claim 15, further comprising: receiving user input to activatedisplaying of the video information; and releasing suspension of theaudio based on the user input to activate the displaying of the videoinformation.
 26. The method of claim 15, wherein receiving comprisescommunicating with the external device after performing a predefinedauthentication procedure with the external device.
 27. The method ofclaim 26, further comprising: creating a control signal to control theexternal device through the display device.
 28. The method of claim 15,wherein the user input comprises a voice command.